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Supplemental Education Services (SES) 
provide extra academic instruction to eligible 
students (i.e., economically disadvantaged 
students who receive free or reduced-price 
lunch [ERL]) beyond the instruction received 
during the school day. 

Based on Title I, Part A of the Elementary and 
Secondary Education Act of 1965 as amended 
by the No Child Left Behind Act of 2001 
(NCLB), Title I schools that have not made 
Adequate Yearly Progress (AYP) for three 
years enter into their second year of school 
improvement and are required to offer ERL 
students SES in addition to existing Title I 
services. Eor additional information on AYP 
see ABCs and AYP results, WCPSS: 2009-10: 
at http://www.wcpss.net/evaluation-research/ 
reports/201 lZ1020abc-ayp09-10.pdf. Eederal 
Title I legislation focuses on improving the 
academic achievement of disadvantaged 
students; thus, students who receive ERL are 
eligible for SES regardless of their 
achievement level (DPI, 2008). Offering ERL 
students additional academic opportunities 
beyond the school day recognizes the economic 
constraints that restrict these students from 
accessing extra learning opportunities in the 
same manner as students from more affluent 
families. 

This report is the second of three reports 
examining SES within WCPSS. The first 
report provided a description of SES within 
WCPSS in terms of the schools, providers, and 
student participants in 2008-09 
and 2009-10 (Supplemental 
Education Services: 2008-09 & 
2009-10 at: http:// 

WAKE COUNTY www wjss.net/e™limdon- 

PUBLIC SCHOOL SYSTEM research/reports/201 1/1 101ses09- 

lO.pdf). This report will focus on 



Major Findings 

Implementation Requirements: SES has 13 

implementation requirements for the Local 
Educational Agency (7), providers (2), and 
parents (4). Nine were fully met, with four 
partially met. 


• Characteristics of the SES Program: Live 
elementary schools — Brentwood, Eox Road, Poe, 
Wendell, and York— provided SES to 508 (36%) 
of the 1,423 students eligible in 2009-10. Parents 
selected 14 of the 47 provider agencies available 
to WCPSS students. Most students received 
tutoring services in reading and mathematics 
twice a week after school. While on average 
groups consisted of five students, group size 
ranged from one to eight students. 


• Short-term and Intermediate Goals: The 

two short-term goals — student enrollment in SES 
and providers pre-assess all participants — were 
met. Three of the six intermediate goals were met 
(progress reports given to parents, minimum of 30 
hours offered to each student, monitoring showed 
implementation occurring) and two additional 
goals were partially met (attendance at SES and 
post-assessments given to completers). 


• Study Conclusions: In 2009-10, the SES 
program was largely implemented with fidelity, 
with some areas needing refinement related to 
communication and monitoring. Monitoring 
reports noted appropriate instruction based on 
learning plans was occurring. 

• Recommendations: Site Coordinators should 
share information generated from tutoring 
services with classroom teachers to improve 
understanding of students’ academic needs. 
Greater collaboration in this area could increase 
impact based on prior research. Ways to ensure 
pre- and post-assessment data are accessible 
should be built into the monitoring system. 
Additionally, the monitoring system should 
include a process for verifying that all parents are 
contacted with regard to excessive student 
absences. 


Implementation Insights reports provide basic 
information on program implementation. These 
reports should be used by program staff and decision- 
makers as guides to determine the future needs and 
direction of training and program implementation. 


Report contribution acknowledgements: WCPSS staff (listed alphabetically): Nancy Baenen, Martina Lowry, Robert Sox, and 
Willi Webb and R. Reichstetter of Crystal Learning Concepts (contractor) 
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the implementation of SES, and the degree to which short-term and intermediate goals were met. The 
third and final report, planned to be completed by June 2011, will focus on student academic 
outcomes in the five elementary schools that participated in SES in 2009-10: Brentwood, Fox Road, 
Poe, Wendell, and York. The number of schools in WCPSS required to offer SES increased to 10 
in 2010-11. 

Methods 

In order to provide a comprehensive examination of the implementation of SES, feedback on SES 
implementation in 2009-10 was gathered from multiple sources — North Carolina’s Department of 
Public Instruction (DPI) documents and data files, WCPSS school and central records and data 
files, staff and provider surveys, and staff interviews. Two online surveys were administered in 
February 2011, one to site coordinators at each of the schools offering SES in 2009-10 and one to 
contacts at each of the provider agencies that serviced WCPSS students in 2009-10 (see Appendixes 
B and C for survey instruments). The site coordinator survey had a response rate of 100%, with all 
five of the site coordinators in 2009-10 responding, and the provider survey had a response rate of 
79%, with 1 1 of the 14 provider agencies responding. Central services Title I staff were also 
interviewed regarding the implementation of SES in 2009-10. The logic model and evaluation 
questions were reviewed with Title I staff (see Appendix A). SES participation data were obtained 
from DPI via an electronic data file generated from the Consolidated Federal Data Collection 
System. 

Evaluation Questions 

This evaluation examines the implementation of SES within WCPSS in 2009-10 and provides a 
description of the degree to which the short-term and intermediate goals of the program were met. 
This report is organized around the following evaluation questions: 

1. Were SES implementation requirements met in 2009-10? 

2. What were the characteristics of the SES programs in which students participated in 2009-10? 

In what subjects were students tutored? 

3. Were SES short-term and intermediate goals met? The short-term and intermediate program 
goals included: 

• students are enrolled in SES; 

• provider pre-assesses all participants; 

• students attend SES sessions; 

• progress reports are given to all parents; 

• a minimum of 30 service hours are offered per child; 

• provider post-assessments are given to completers; 

• 80% of learning plan goals are met; and 

• program monitoring shows implementation is occurring. 
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Question 1: Were SES implementation requirements met in 2009-10? 

The provision of SES requires the involvement of multiple stakeholders — states, i.e.. North Carolina, 
Local Education Agencies (LEAs) i.e., WCPSS, provider agencies, and parents. For each group of 
stakeholders there are requirements attached to the provision of SES as summarized in Table 1. The 
degree to which each requirement was met is also shown (see Attachment D for more details). 


Table 1 

SES Implementation Responsibilities and Status 


Entity 

Category 

Status 

Comment 

LEA 

Identify eligible students 


1,423 ERL students attended schools offering SES 
and were thus eligible for SES in 2009-10. 

Notify parents about SES availability 


WCPSS sent notification letters and schools held 
provider fairs to notify parents (see Appendix E). 

Identify eligible students to receive 
services (if not all students can be served) 


No prioritizing was necessary: all eligible students 
requesting SES were able to receive services. 

Meet 20% funding obligation 


In 2009-10, WCPSS set aside $4,138,186.80 (20%) 
of the $20,690,934 received in Title I funding for 
the provision of school choice/SES. 

Enter into an agreement with a provider 
selected by parents of an eligible student 

y^ 

An agreement between the LEA and each provider 
was signed. The agreement form met U.S. Dept, of 
Education and DPI requirements. 

Ensure that parents are notified by the 
provider if their child is not attending 
regularly 

Partially 

Met 

81.8% of providers reported they were monitored 
by the school site coordinator, WCPSS 
administrator, and/or DPI staff (see Eigure 1). 

Obtain parent feedback on satisfaction with 
SES services 

y^ 

A parent survey was administered by SERVE and 
distributed by LEAs. 

Provider 

Attend to logistics of agreement with LEA 

y^ 

Overall, site coordinators and providers reported 
favorably regarding SES session logistics. 

Provide instruction based on agreement 
with LEA 

Partially 

Met 

Post-assessments were not given to all completers. 
Nearly all reported learning plans were developed 
based on pre-assessments, but LEA and parent or 
guardians were not always involved. 

Parent 

Select a provider from the State- approved 
list that is in or near the LEA 

y^ 

Parents selected 14 of the 47 state approved 
providers offering SES within WCPSS. 

With the LEA and the selected provider, 
develop and identify specific academic 
achievement goals for the student, 
measures of student progress, and a 
timetable for improving achievement 

Partially 

Met 

Providers reported collaboration with parents did not 
always occur as required. 

Ensure that their child attends the SES 
sessions in which he or she is enrolled 

Partially 

Met 

70.6% of students received 26 or more hours of 
tutoring; > one-third attended 100% of sessions. 

Other desired responsibilities 

• Support provider and LEA efforts to work 
with their child to attain achievement 
goals 

• Encourage their child to succeed 

• Provide feedback on satisfaction with SES 

y^ 

Overall, 10,936 parents in NC provided feedback on 
the parent survey. The survey did not address 
whether parents encouraged their child to succeed, 
and therefore was not measured. 


Note: DPI contracted with the SERVE center to administer parent satisfaction surveys. Surveys were sent to LEA/Charter School 
representatives at 82 districts and Charter Schools and representatives sent the survey home with SES-enrolled students. 

Data Source: U.S. Department of Education (2009). No Child Left Behind: Public School Choice Non-Regulatory Guidance. 

Retrieved January 20, 2011, from http://www2.ed.gov/policy/elsec/guid/suppsvcsguid.doc; 2009-10 SES Implementation Site 
Coordinator and Provider Surveys; Title I staff interviews. 
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State and LEA Requirements 

At the state and LEA level, DPI was required to identify qualified 
agencies, both public and private, available for parent selection and 
WCPSS was required to identify eligible students and notify parents of 
these eligible students regarding their provider options/ The first SES 
report in this series (Supplemental Education Services: 2008-09 & 

2009-10 at: http://www.wcpss.net/evaluation-research/reports/ 

201 1/1 101ses09-10.pdf) found that the SES programs for WCPSS in 
2008-09 and 2009-10 were in compliance with federal guidelines to 
make available an SES program, enroll only ERL students, and to use 
state-approved vendors to deliver the program. Additional LEA 
requirements included obtaining parent feedback on their satisfaction 
with SES programs and ensuring parents are notified by the provider if 
their child is not attending regularly. Parent surveys were conducted 
in 2009-10 and will be addressed in the parent requirements section of 
this report. 

While two of the 1 1 providers (18.2%) did not report monitoring by WCPSS or DPI staff, the 
remaining nine (81.8%) providers reported being monitored by WCPSS and/or DPI staff. WCPSS 
Title I staff reported SES sessions were routinely monitored by school site coordinators, WCPSS 
representatives, and DPI representatives. Monitoring included: 

• on-site visits; 

• observations of tutoring sessions; 

• reviews of learning plans; and 

• tutor interviews. 

Monitoring of SES programs is essential to ensuring providers notify parents of students with poor 
attendance. In addition to self-monitoring conducted by provider administrators or designees: 

• nine of the 1 1 (81.8%) providers reported that their SES program was monitored by the school 
site coordinator, 

• three out of the 1 1 (27.3%) by WCPSS administrator, and/or 

• three out of the 1 1 (27.3%) by DPI staff. 

The two providers who reported that they were not monitored by WCPSS or DPI staff may have 
been unaware of the monitoring of their learning plans performed after submission. 
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Title I staff reported 
SES sessions were 
routinely monitored 
by school site 
coordinators, WCPSS 
representatives, and 
DPI representatives, 

and 

81 . 8 % of providers 
reported being 
monitored by WCPSS 
and/or DPI staff. 


' Qualified SES “must be high quality, research-based, and specifically designed to increase student academic 
achievement” (U.S. Department of Education, 2009). 
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Figure 1 

SES Providers Report on Who Monitored SES Sessions 



Provider administrator School site coordinator WCPSS administrator DPI staff 

or designee 

Note: 1 . A^=l 1 provider contacts who responded to the survey. 

2. Categories are not mutually exclusive. 

Data Source: 2009-10 SES Implementation Provider Survey. 

Provider Requirements 

Provider responsibilities included ensuring curriculum materials 
were consistent with the NC Standard Course of Study; developing 
an agreement between the district/school, the provider, and the 
parents outlining the student’s specific achievement criteria and 
goals; and providing parents with progress reports (DPI, 2008). In 
order to assess the degree to which providers met SES requirements, 
this section examines student learning plans, communication across 
groups of stakeholders, and session logistics. 

Student Learning Plans 

Learning plans matching student academic needs were to be developed collaboratively between 
providers, WCPSS staff, and parents. The learning plan templates provided by DPI and WCPSS 
included both pre-assessment and prior student achievement (see Appendix F for a WCPSS 
learning plan template). Title I staff reported that individual learning plans built upon the NC 
Standard Course of Study competencies and matching students’ needs were developed for served 
students. Almost all (90.9%) provider contacts who responded to the survey reported that both pre- 
assessment results and the consideration of individual student needs were used in the development 
of student learning plans 100% of the time (see Figure 2). 


Student learning 
plans were based on 
the NCSCS and pre- 
assessment nearly all 
of the time, but 
required 

collaboration with 
parents did not 
always occur. 
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WCPSS Title I staff reported the learning plan development process required providers to submit 
individualized learning plans — developed following the administration of the pre-assessment — to 
the Title I office for approval. Learning plans were reviewed by Title I staff and either approved or 
disapproved and returned to the provider for modifications. In addition to the development process, 
WCPSS staff also monitored SES sessions to determine if providers were following student 
learning plans. Provider contacts, however, did not always report working with WCPSS staff to 
develop learning plans. This mismatch may be the result of a different definition of collaboration, 
with some providers expecting more active involvement in the development phase from school or 
central WCPSS staff. 

• Only one of the 1 1 respondents (9.1%) reported that they worked with a WCPSS representative 
100% of the time; 

• Three out of 1 1 respondents (27.3%) reported they worked with a WCPSS representative at 
least half of the time, but did not always do so; and 

• Seven out of 1 1 respondents (63.6%) reported they worked with a WCPSS representative less 
than 50% of the time (27.3% reported not doing it at all). 

While working with parents to develop student learning plans was reported as a more common 
practice, providers reported collaboration with parents did not always occur: three out of 1 1 
(27.3%) reported this occurred less than 50% of the time. 

Figure 2 

Provider Report on Development of Learning Plans in 2009-10 



Note: N= 1 1 provider contacts who responded to the survey. 

Data Source: 2009-10 SES Implementation Provider Survey. 
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Communication 

Site coordinators most often reported that providers shared student learning plans (five out of five 
or 100%) and attendance records (four out of five or 80.0%) with them (see Figure 3). Fewer site 
coordinators reported receiving student progress reports (three out of five or 60.0%) and/or pre- and 
post-assessment results (two out of five or 40.0%). It should be noted that in each instance, site 
coordinators were less likely to share the information received from providers with the students’ 
teachers. Indeed, although three of the five site coordinators reported receiving student progress 
reports from provider agencies, none shared this information with the students’ teachers. Title I 
staff reported that site coordinators were expected to share student progress with teachers upon 
request. We have no data to inform why site coordinators shared learning plans and not progress 
reports. 


Figure 3 

Site Coordinators’ Report on Communication Regarding SES Students’ Progress in 2009-10 


c 

cu 

■D 

c 

o 

Q. 




u 

Student 
learning plans 

Student pre- 
and post- 
assessment 
results 

Student 

progress 

reports 

Student 

attendance 

records 

■ Provider shared with Site 
Coordinator 

5 

2 

3 

4 

Site Coordinator shared with 
Teacher 

4 

1 

0 

2 


Note: N-5 2009-10 site coordinators. 

Data Source: 2009-10 SES Implementation Site Coordinator Survey. 


Providers reported that they always shared student progress reports and pre- and post-assessment 
results with parents and 10 out of 1 1 or 90.9% also reported sharing this information with site 
coordinators; this was much higher than reported by site coordinators (see Figure 4). 
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Figure 4 

Providers’ Report on Communication Regarding SES Students’ Progress in 2009-10 
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u 

Pre-and post-assessment 
results 

Progress reports 

■ Parents or guardians 

11 

1 

■ School site coordinator 

10 

11 

■ Teacher 

3 

0 

■ School principal 

0 

0 

■ District central office staff 

3 

3 

■ Tutors 

1 

1 

■ SERVE 

0 

1 


Note: In 2009-10, DPI contracted with the SERVE Center to evaluate the performance of 

SES providers using three criteria: student attendance, parental satisfaction, and 
academic achievement. 

Data Source: 2009-10 SES Implementation Provider Survey. 


In addition to the creation of learning plans and notification of 
student progress, another area in which communication was an 
important consideration was that of student attendance. WCPSS was 
required to ensure providers notified parents if their child was not 
attending SES sessions regularly. 

Approximately two-thirds (63.6%) of provider contacts reported 
always contacting parents of students not attending SES sessions 
regularly; 27.3% reported this occurred often; and 9.1% reported it 
occurred sometimes. 


Approximately two- 
thirds (63.6%) of 
provider contacts 
reported always 
contacting parents of 
students not 
attending SES 
sessions regularly. 

J 


Session Logistics 

Overall, site coordinators and providers reported favorably regarding SES session logistics. The 
vast majority (four out of five or 80.0%) of site coordinators and almost all (10 out of 1 1 or 90.9%) 
of the contact persons at each of the provider agencies who responded to the survey reported that 
sessions always took place as scheduled and started and ended on time (see Eigures 5 and 6). 
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Figure 5 

Site Coordinator Report on Frequency of SES Activities in 2009-10 
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Always 

Often 

Sometimes 

Never 

■ SES session procedures were 
efficiently organized for students 
to attend scheduled sessions. 

4 

1 

0 

0 

■ Site coordinator participated in 
monitoring SES sessions. 

4 

1 

0 

0 

■ SESsessionstookplace as 
scheduled. 

5 

0 

0 

0 

■ SES sessions started on time. 

4 

1 

0 

0 

SES sessions ended on time. 

4 

1 

0 

0 


Data Source: 2009-10 SES Implementation Site Coordinator Survey. 


Figure 6 

Provider Agency Contacts Report on Frequency of SES Activities in 2009-10 
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Always 

Often 

Sometimes 

Never 

■ OurSES sessions took place as 
scheduled. 

10 

1 

0 

0 

OurSES sessions started on time. 

11 

0 

0 

0 

OurSES sessions ended on time. 

10 

1 

0 
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Data Source: 2009-10 SES Implementation Provider Survey. 
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All of the provider agencies offered students 30 or more hours of tutoring — 13 providers offered 
students the required 30 hours of services while one agency (Sterling Learning Centers) offered 
students 40 hours of services. Thus, all of the 508 students enrolled in SES in 2009-10 were offered 
30 hours fulfilling the SES requirement. 

No common implementation concerns were shared across site coordinators. One site coordinator 
reported concern over staffing while another mentioned that scheduling was an issue. Another 
implementation concern mentioned was the timeliness of paperwork from the providers. One site 
coordinator reported that some provider agencies filed Facility Use forms late, requiring schools 
and Title I staff to reassign students at the last minute (in some cases the day prior to tutoring 
sessions beginning). In some cases, “providers, especially those out-of-area, did not even realize 
that their local reps had not filed the paperwork.” A provider contact also stated, “facilities usage 
and the deadlines set for the SES program were not congruent and created a communication gap.” 

Sylvan Learning Center, the largest SES provider in 2009-10, was originally expected to serve 
students on-site, but switched to off-site services after parents had already selected this provider. 
One site coordinator reported, “Sylvan ended up tutoring students several months after the parents 
had anticipated the tutoring to begin.” Logistical concerns regarding Sylvan Learning Center were 
also shared by Title I staff who reported scheduling had been an issue with this provider. Other 
implementation concerns mentioned by site coordinators included: completion of attendance reports 
by providers, students being picked up in a timely manner, and notification letters to parents. Three 
of the five site coordinators reported parents were notified in time for their child to attend the first 
scheduled SES session. 

Provider contacts had two shared concerns: scheduling (36.4%) and staffing (54.5%). Four of 1 1 
providers said they had issues with scheduling for SES program implementation; three specified the 
following problems: 

• “getting accurate student data (e.g., accurate telephone numbers, accurate home address)”; 

• “unable to get accurate contact information for students”; and 

• “timeliness procedures and lack of understanding by Facilities [ Communities in Schools] 
Department with its bureaucratic layers of review of SES provider's submitted facility use 
forms. ” 

Six of 1 1 providers said they had issues with staffing for the SES program implementation; five 
specified with the following: 

• “finding quality and passionate staff”; 

• “rarely able to recruit teachers from school”; 

• “not sufficient time to hire, train and do background checks of teachers between district 
releasing student enrollment list and start date of tutoring” ; 

• “finding committed staff; teachers from school are least likely willing to serve as dependable, 
reliable tutors to follow SES procedures for which Provider is held accountable” ; and 

• “schedules of certified teachers in school did not allow hiring them as tutors. ” 
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Parent Requirements 

There were also requirements associated with parents including: 

• selecting a provider from the state-approved list that is in or near the LEA; 

• working with the LEA and the selected provider to develop and identify specific academic 
achievement goals for the student, measures of student progress, and a timetable for improving 
achievement; 

• ensuring that their child attends the SES sessions in which he or she is enrolled; 

• supporting provider and LEA efforts to work with their child toward attaining achievement 
goals; 

• encouraging their child to succeed; and 

• providing feedback on satisfaction with SES services. 

In 2009-10, 14 of the 47 providers offering SES within WCPSS were selected by parents to provide 
SES services to WCPSS students. Providers did report working with most parents to develop 
student learning plans; 72.8% of providers reported this always occurred or occurred with at least 
half of the parents (see Figure 2). While parent support of provider and LEA efforts and level of 
encouragement given to their child was not directly observable, attendance (i.e., ensuring students 
attend regularly) could be considered a proxy for parental support. Thus, the final parental 
requirements examined were student enrollment and attendance and parent feedback. 

Student Enrollment and Attendance 

Although the number of hours of tutoring received by SES 
participants ranged from 1.5 to 38 hours, over two-thirds (70.6%) of 
students did attend 26 or more hours of tutoring (see Figure 7). More 
than one-third (35.6%) of students attended 30 or more hours and an 
additional 35.0% attended between 26 to 29 hours. Of course, this 
means 29.3% of attendees received fewer than 26 hours of tutoring. 

Attendance rates were generally strong; the average attendance rate 
was 87.2%. Indeed, Figure 8 shows that the vast majority (84.0%) of 

students attended 80% or more of their sessions. However, 73% of providers reported attendance 
was an issue. Among these providers, most listed transportation as the reason for the difficulties 
associated with attendance. Transportation was not provided by WCPSS; thus, all transportation 
was arranged between parents and providers. Provider comments regarding poor attendance 
included statements such as: 

• '"Most attendance issues are due to parents not having access to transportation to pick up 
kids 

• “Students did not have transportation and 

• “Lack of transportation or conflicts with parent work schedules." 


“Most attendance 
issues are due to 
parents not having 
access to 

transportation to pick 
up kids ” 

(provider contact) 
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Figure 7 

Percentage of Students by Number of Hours Served in SES 


2009-10 



Hours Received 

Number 

Percent 

1 to 7 

15 

3.0% 

8 to 13 

16 

3.2% 

14 to 19 

23 

4.5% 

20 to 25 

95 

18.7% 

26 to 29 

178 

35.0% 

30 or more 

181 

35.6% 

Total 

508 

100.0% 


Data Source: Data file obtained from DPI via an electronic data file generated from the Consolidated Federal 
Data Collection System. 


Figure 8 

SES Students’ Attendance Rates by Range 
2009-10 



Attendance Rate 

Number 

Percent 

less than 20% 

7 

1.4% 

20% to 39% 

18 

3.5% 

40% to 59% 

14 

2.8% 

60% to 79% 

42 

8.3% 

80% to 99% 

249 

49.0% 

100% 

178 

35.0% 

Total 

508 

100.0% 


Data Source: Data file obtained from DPI via an electronic data file generated from the Consolidated Federal 
Data Collection System. 
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Figure 9 displays enrollment and attendance rates by provider agency. The number enrolled by site 
varied from three at two sites to 155 at Sylvan Learning Center. Seventy percent of students 
attending Sylvan in 2009-10, had 100% attendance. Attendance in the range of 80-99% was most 
common by provider. It should be noted that since six providers served fewer than ten students 
each; percentages based on small numbers of students should be viewed with caution. 


Figure 9 

Attendance Rates by SES Provider 
2009-10 
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Sterling 

Learning 
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n=5 
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Learning 

Solutions 

n=14 

■ 100% 

13% 

21% 

33% 

88% 

33% 

100% 

33% 

32% 

70% 

14% 

15% 

8% 

0% 

0% 

■ 80% to 99% 

70% 

66% 

58% 

0% 

67% 

0% 

67% 

45% 

14% 

71% 

63% 

78% 

60% 

86% 

■ 60% to 79% 

11% 

10% 

8% 

13% 

0% 

0% 

0% 

13% 

5% 

14% 

15% 

2% 

0% 

14% 

■ 40% to 59% 

2% 

1% 

0% 

0% 

0% 

0% 

0% 

11% 

1% 

0% 

0% 

8% 

40% 

0% 

■ 20% to 39% 

3% 

2% 

0% 

0% 

0% 

0% 

0% 

0% 

6% 

0% 

5% 

4% 

0% 

0% 

■ less than 20% 

0% 

0% 

0% 

0% 

0% 

0% 

0% 

0% 

4% 

0% 

2% 

0% 

0% 

0% 


Note: Percentages based on small numbers of students should be viewed with caution. 

Data Source: Data file obtained from DPI via an electronic data file generated from the Consolidated Federal Data 
Collection System. 
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Implementation of SES 
Parent Feedback 

In 2009-10, SERVE Center (under contract with DPI) evaluated the 
performance of SES providers using three criteria: student 
attendance, parental satisfaction, and academic achievement. 

Parent satisfaction was measured via parent surveys developed 
utilizing DPI approved survey questions (Williams, Meli, Poole, & 

Amwake, 2010). Parent survey data were used as an indicator to 
rate provider standing in accordance with the SES policy. A 
composite score for providers was generated based on the weighted 
average of the three criteria (student attendance, parental satisfaction, and academic achievement); 
therefore, the parent survey was not analyzed separately. District level data were reported but only 
for selected questions related to the provider selection process and communication. The seven 
questions pertaining to parent satisfaction with WCPSS programs are presented in Appendix H. 
Parent survey results in 2009-10 indicated: 

• Overall, parents were satisfied with WCPSS in regards to provider selection and 
communication. The percentage of parents with positive responses ranged from 71.9% to 
93.5% for the seven questions. 

• The vast majority of parents reported that the tutoring services had improved their child’s 
academic skills (86.6%) and they would have their child participate in SES in the next year 
(91.8%). 

Question 2: What were the characteristics of the SES programs in which students 
participated in 2009-10? In what subjects were students tutored? 

In 2009-10, five WCPSS elementary schools offered SES: 

Brentwood, Fox Road, Poe, Wendell, and York. SES was offered 
in two rounds: the first round began in November 2010 and the 
second in February 2011. WCPSS notified parents of SES 
opportunities through letters and schools held provider fairs. 

Site coordinators (100%) and provider contacts (90.9%) reported 
students received tutoring services twice a week after school. Most 
(72.7%) provider contacts reported the typical session was 1.5 hours long. Site coordinators 
reported the typical group size ranged from three to eight students.^ All of the 1 1 provider contacts 
who responded to the survey reported providing services on-site (at the school). However, Sylvan 
(the largest provider in 2009-10) served students off-site.^ 


Most students 
received tutoring 
services in reading 
and mathematics 
twice a week after 
school. 

J 
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% 

Overall parents were 
satisfied with WCPSS 
in regards to provider 
selection and 
communication. 


^ Due to questions regarding responses, follow-up calls were conducted. Four of the five site coordinators were 
contacted and their data are discussed here. Follow-up calls were unsuccessful in reaching the other site coordinator. 
^ The contact from Sylvan did not respond to the provider survey. 
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In 2009-10, 47 vendors offered SES in WCPSS; 14 were selected by parents to provide tutoring 
services. While two of the providers — ATS Project Success and Learning & You — provided 
services online, the remainder offered face-to-face instruction. The percentage of students 
participating in SES by provider ranged from 31% attending Sylvan Learning Center to <1% 
attending It’s Simply English and utilizing ATS Project Success (three students each), see Table 2. 


Table 2 

Supplemental Educational Services Participation by Provider 

2009-10 


Provider 

Number 

Percent 

Academic Achievers/S &L Consultants 

61 

12.0% 

Academics Plus, Inc. 

82 

16.1% 

Achieve Success Tutoring by University Instructors 

12 

2.4% 

AIM by Salient Learning 

8 

1.6% 

Allied Academics 

9 

1.8% 

ATS Project Success 

3 

0.6% 

It’s Simply English 

3 

0.6% 

Learning & You Contractors, LLC 

38 

7.5% 

Master Mind Prep Learning Solutions, Inc. 

7 

1.4% 

Measurement Incorporated (Accelerated Achievement ) 

60 

11.8% 

Shaw University/Historically Minority Colleges & 
Universities Consortium of NC 

51 

10.0% 

Sterling Learning Center, Inc. 

5 

1.0% 

Sylvan Learning Center (Ace It) 

155 

30.5% 

Village Learning Solutions 

14 

2.8% 

Total 

508 

100% 


Note: Data based on complete data file from DPI; thus, numbers and percentages do not 

correspond to those reported in prior SES report (Supplemental Education Services: 2008- 
09 & 2009-10 at: http://www.wcpss.net/evaluation-research/reports/2011/1101ses09- 
lO.pdf) 

Data Source: Data file obtained from DPI via an electronic data file generated from the Consolidated 
Federal Data Collection System. 
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Subjects Tutored 

In 2009-10, all but one of the 14 SES providers selected (It’s Simply English, which focused only 
on reading) offered SES tutoring in reading and mathematics (see Eigure 10). Of the 508 students 
participating in SES in 2009-10, almost all (98%) were tutored in reading and mathematics. Across 
providers eight students received tutoring in reading only and one student in mathematics only, and 
the remaining 499 students received tutoring in both reading and mathematics. 

Figure 10 

Number of Students Served in SES by Provider Agency and Subject 

2009-10 
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■ Reading 

0 

0 

0 

0 

0 

0 

3 

5 

0 

0 

0 

0 

0 

0 

■ Math 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 
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61 

82 

12 

8 

9 

3 

0 

33 

7 

60 

51 

5 

154 

14 



Note: Number of students by provider is based on the 508 students participating in SES in 2009-10; thus, the 
number of students shown here reflects a higher number than reported in the previous report. 
Data Source: Data file obtained from DPI via an electronic data file generated from the Consolidated Federal 
Data Collection System. 
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Question 3: Were SES short-term and intermediate goals met? 

Table 3 displays the status of each of the short-term and intermediate goals of the SES program. 
There were two short-term goals identified within the logic model (see Appendix A) — students 
enrolled in SES, and providers pre-assess all participants. 

• The first short-term goal of student enrollment in SES was met, with 508 eligible students 
participating at the five schools required to offer SES in 2009-10. 

• The second short-term goal was met. Based on a random sample of individual student learning 
plans all student records sampled had a pre-assessment score recorded on the student’s learning 
plan. However, the centrally collected data file was incomplete. While the vast majority 
(88.8%) of students with complete data did have a pre-assessment score recorded, 1 1.2%, or 36 
students did not. 

Three of the six intermediate goals were met. Two additional goals were partially met, and one 
could not be assessed due to a lack of data (see Table 3). 

• Overall, monitoring revealed implementation occurred, provider agencies reported sharing 
progress reports with parents, and all students were offered 30 or more hours of service; thus, 
these intermediate goals were met. 

• However, two additional intermediate goals were only partially met. 

► While students attended regularly, with an average attendance rate of 87.3%, 29.4% 
attended less than 26 hours of services. 

► Based on available data, 81.5% of students with 25 or more hours of service received had a 
post-assessment score. Providers reported reasons for students not being assessed were 
attrition or absences. One provider reported, “students who did not attend the last session 
were not post-tested. Scheduling make-ups would require paperwork for facility use. It is 
not feasible to consider make-up sessions.” 

• Finally, the degree to which the intermediate goal of 80% of individual student learning plans 
being met could not be determined due to a lack of data. In 2009-10, while there was a process 
of submitting learning plans, there were no procedures in place to review learning plans and 
determine the degree to which students’ goals were met through SES. 

The long term goal for the year of improved student achievement will be evaluated in the third and 
final report in this series due out in June 2011. 
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Table 3 


Status of Supplemental Education Services Goals 


Level 

Goal 

Status of Goal 

Short-term 

Students enrolled in SES. 

Met 

508 ERL students enrolled in 2009-10 

Provider pre-assesses all participants. 

Met 

100% of sampled student learning plans 
included a pre-assessment score 


Attendance at SES 

(26 out of 30 hours attended) 

Partially Met 

70.6% of students received 26 or more hours 
of tutoring 


Progress reports given to all parents 

Met 

100% of providers 


Minimum of 30 service hours offered per child. 

Met 

Intermediate 

Provider post-assessments given to completers. 

Partially Met 

81.5% (159 of the 195) students with 25 or 
more hours of service received had a post- 
test score 


80% of learning plan goals met. 

Unknown 


Program monitoring shows implementation 
occurring. 

Met 


All sub-groups met reading and mathematics growth 
targets (academic change) on EOG. 

Will be examined in Report 3 

Long-term 

A higher percentage of SES participants met reading 
and mathematics growth targets (academic change) 
than prior year before receiving SES (grades 4 and 5). 

Will be examined in Report 3 


Higher percentage of SES participants proficient on 
EOG than prior year before receiving SES (grade 4 
and 5 only). 

Will be examined in Report 3 


Data Source: 


Data file obtained from DPI via an electronic data file generated from the Consolidated Federal Data 
Collection System and 2009-10 SES Implementation Site Coordinator and Provider Surveys. 
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CONCLUSIONS AND DISCUSSION 

Fidelity of implementation is an important contributor to program success. Short-term and 
intermediate goals were set to ensure SES was implemented with fidelity, requirements were met, 
and to improve the likelihood of reaching the long-term goal of improved student achievement by 
the end of the school year. The provision of SES is a NCLB requirement for schools in their second 
year of school improvement; thus, this implementation evaluation reviewed both the degree to 
which requirements were satisfied and short-term and intermediate goals were met. In 2009-10, 
most of the LEA, provider, and parent requirements were met. Both short-term goals and half of 
the intermediate goals were also met; however, goals involving post-assessments and attendance 
were only partially met. 

Short-term and intermediate goals included: enrolling students in SES, monitoring student 
attendance, developing individual student learning plans, providing instruction specific to student 
needs, and assessing students on provider pre- and post-assessments. As with most programs, the 
enrollment of appropriate students is key to the successful implementation of SES. Although the 
short-term goal of student enrollment in SES was met with 508 students attending SES in 2009-10, 
only 36% of eligible students participated in the SES program, which left room for increased 
participation within this program. 

Overall, the SES program was largely implemented with fidelity, with some areas for refinement 
related to communication and monitoring. Although four of the five (80%) site coordinators 
reported sharing learning plans with classroom teachers, only one reported sharing pre- and post- 
assessments, and none reported sharing student progress reports. The reason for this is unclear. 
While parents give permission for the teachers to be contacted, site coordinators may be unclear 
about what and when to share information with teachers. 

There were conflicting reports regarding the monitoring of SES in 2009-10. Two of the 1 1 
providers who responded to the survey reported they had not been monitored by either WCPSS or 
DPI staff. Yet, DPI and WCPSS site coordinators and central services staff reported monitoring 
provider agencies. Additionally, document review of Title I files revealed monitoring reports 
(which dealt with issues such as the match of tutoring services provided and student learning plans) 
were filed (see Appendix G for WCPSS monitoring form template). 

Additionally, working with the LEA to create learning plans also revealed conflicting reports 
between provider contacts and Title I staff. While Title I staff described a process of reviewing 
submitted learning plans in addition to site visits and interviews, provider contacts did not always 
recognize working with WCPSS staff to develop learning plans. Indeed, only one of the 1 1 
respondents reported that they worked with a WCPSS representative 100% of the time. This is 
likely because the process of reviewing learning plans within the Title I office did not require 
follow-up to providers if learning plans were approved without modifications. Providers were only 
contacted if student learning plans were not approved and required modifications; thus, the provider 
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may have viewed the submission of the learning plans as strictly an administrative task and may 
have been unaware of the review process. Providers may also have been hoping for earlier 
involvement by school staff. 

One additional implementation concern — expressed by more than half of the provider contacts — 
was the difficulty of securing qualified tutors. Providers stated that hiring teachers from the school 
was difficult if not logistically impossible due to schedules conflicts. They also mentioned that it 
was difficult to find “quality,” “committed,” and “passionate” staff which was probably exacerbated 
by their inability to hire teachers from the students’ schools. 

In 2005-06, Hodge Road Elementary was the first WCPSS school required to offer SES. Looking 
back to the evaluation of the 2005-06 program at Hodge Road, implementation concerns included 
problems associated with session logistics and the inappropriate use of remedial curriculum 
materials with those ERL students who began the program at or above grade level (Paeplow & 
Baenen, 2006). Concerns reported in the evaluation of the 2005-06 program at Hodge Road were 
either not found or were less evident in 2009-10. The logistical problems such as absent or late 
tutors were not reported in 2009-10. Furthermore, in 2009-10 students were offered tutoring 
services based on individualized learning plans developed utilizing prior student achievement. 

While lack of transportation and procedures related to facility use were mentioned as logistic 
concerns, there were no concerns shared across site coordinators and overall site coordinators 
reported favorably regarding session logistics. It appears that prior experiences and reports 
contributed to more specific requirements, greater use of pre-assessments, stronger individual plans, 
and increased monitoring. 

SES is focused on equity for disadvantaged (i.e., ERL) students, and is designed to improve the 
achievement of FRL students regardless of their achievement level. Thus, in 2009-10, to ensure 
students were served at the appropriate achievement level, individual learning plans were 
developed. 


RECOMMENDATIONS 

In 2009-10, SES was implemented with fidelity within WCPSS with minor exceptions. While 
overall the implementation was good, there were a few areas for improvement. At this point, we 
have the following recommendations for improved communication, monitoring, and increased 
participation in the SES program in WCPSS: 

• Site Coordinators share information with classroom teachers. The sharing of information 
between the classroom and tutoring services will improve both teachers’ and tutors’ 
understanding of a student’s academic needs; thus, coordinators should share information 
generated from tutoring services with classroom teachers. Parent consent could be solicited to 
allow the sharing of information with classroom teachers and ensure confidentiality 
requirements are observed. 
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• Increase collaboration and clarify and refine existing monitoring procedures. Title I staff 
should communicate monitoring procedures related to the development and submission of 
learning plans to the providers. Inconsistent reports from Title I staff and provider contacts 
suggest procedures are in place to review learning plans, but the purpose of these procedures 
may not be understood by providers. Given that collaboration is a required element of the 
development of learning plans, any procedure that satisfies this requirement should be 
understood by all stakeholders. Teachers, for example, fill in background information on 
student needs. Staff should review the monitoring and learning plan forms to see if they need to 
be more specific regarding issues such as differentiation. Additionally, in 2009-10 there was no 
WCPSS procedure in place to monitor that individual goals within students’ learning plans were 
being met. 

In 2010-1 1, an electronic data collection process was implemented which allowed for the 
electronic submission of learning plans. The purpose of this submission process should be 
communicated to all stakeholders, and the degree to which student goals were met should also 
be captured either within this system or with an alternative method. 

• Monitor providers to ensure that parents of students not attending are contacted. Ensuring 
providers contact parents of students not attending regularly is currently an LEA requirement. 
Given this, the LEA monitoring process should include questions about attendance and vendor 
follow-up on absences. Vendors could be asked to maintain documentation of attempts to 
contact parents of students and to forward these to the Title I office or site coordinators (after a 
certain number of absences). Long-term, it seems this requirement would be better placed as a 
provider requirement, and district staff could lobby for this change. 

• Collect pre- and post assessment results centrally and post-assess program completers. 
Although a sample of individual student records indicated student learning plans included pre- 
assessments, this information should be consistently captured centrally. In 2009-10, centrally 
collected data showed 88.8% (286 of the 322) of students with assessment data had a pre- 
assessment score; however, 100% of student learning plans sampled included pre-assessment 
data. Given this is a SES requirement, this data should be captured centrally perhaps via the 
newly created Title I electronic system. In terms of post-assessments, 81.5% (159 of the 195) 
students with 25 or more hours of service had a score; therefore, 19.5% of students who 
completed greater than 80% of the program were not administered a post-assessment. Program 
attrition and absences on the day of testing were noted as reasons for the lack of post- 
assessments for some students. Alternative make-up sessions (either at the school or the 
provider agency), could be offered to students missing the last day of tutoring. Additionally, 
over one-third (36.6%) of served students did not have assessment data available centrally; thus, 
a process of collecting this data centrally would ensure requirements associated with pre- and 
post-assessments were monitored and in turn met. 
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Appendix A 

Supplemental Education Services Logic Model 
2008-09, 2009-10 and 2010-11 


Need: “Title I, Part A of the Elementary and Secondary Education Act of 1965 (ESEA), as reauthorized by the No Child Left Behind 
Act of 2001 (NCLB), calls for parents of eligible students attending Title I schools that have not made adequate yearly progress 
(AYP) in increasing student academic achievement for three years to be provided with opportunities and choices to help ensure that 
their children achieve at high levels. SES provide extra academic assistance for eligible children (i.e free or reduce-priced lunch 
recipients). Students from low-income families who are attending Title I schools that are in their second year of school improvement 
(i.e., have not made AYP for three years), in corrective action, or in restructuring status are eligible to receive these services.” 

INPUTS 


STRATEGIES 


OUTCOMES - IMPACT 

Short-Term Intermediate Long-Term 

Beginning of SES Cycle End of SES Cycle End of Year 

SES providers identified 
by the NCDPI (i.e., 
community-based 
organizations). 

Schools not making AYP 
identified by DPI. 

Parents provided 
opportunity to select 
provider. 

Free or reduced-price 
lunch recipients 
identified. 

Individualized learning 
plans created. 

• Additional educational 
services beyond the 
instruction provided 
during the school day 
(i.e., academic 
assistance such as 
tutoring, remediation 
and other supplemental 
academic enrichment). 

• Services consistent 
with the content and 
instruction utilized by 
the district and aligned 
with the NCSCS and 
achievement standards. 

• Students enrolled in SES. 

• Provider pre-assesses all 
participants. 

• Attendance at SES 

• Progress reports 
given to all parents 

• Minimum of 30 
service hours per 
child. 

• Provider post- 
assessments given 
to completers. 

• 80% of Learning 
plan goals met. 

• Program 
monitoring shows 
implementation 
occurring. 

• All sub-groups met reading and 
mathematics growth targets 
(academic change) on LOG. 

• A higher percentage of SES 
participants met reading and 
mathematics growth targets 
(academic change) than prior 
year before receiving SES 
(grade 4 and 5 only). 

• Higher percentage of SES 
participants proficient on LOG 
than prior year before receiving 
SES (grade 4 and 5 only). 


Data Source: U.S. Department of Education. (2009). Supplemental Educational Services Non-Regulatory Guidance. 
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Appendix B 

^ zoomerang 

2009-10 Supplemental Education Services (SES) Implementation 
School Site Coordinator Survey 

Created: February 09 201 1, 8:49 AM 
Last Modified: February 21 201 1, 9:01 PM 
Design Theme: Basic Blue 
Language: English 
Button Options: Labels 
Disable Browser “Back” Button: False 


2009-1 0 Supplemental Education Services (SES) ImplementationSchool Site Coordinator Survey 



WAKE COUNTY 

PUBLIC SCHOOL SYSTEM 


Page 1 - Question 1 - Choice - One Answer (Bullets) 

What percent of the parents of SES students at your school were notified in a timely manner so that their child could 
attend the first scheduled SES session? 


O 100% 

O 50% or more 
O Less than 50% 
O None 


Page 1 - Question 2- Choice - Multiple Answers (Bullets) 

Please indicate which of the following items the providers typically shared with you. (Select all that apply). 


□ Student learning plans 

□ Student pre- and post- assessment results 

□ Student progress reports 

□ Student attendance records 


Page 1 - Question 3- Choice - Multiple Answers (Bullets) 

Please indicate which of the following items you typically shared with the SES students' teachers. (Select all that apply). 


□ Student learning plans 

□ Student pre- and post-assessment results 

□ Student progress reports 

□ Student attendance records 
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Page 1 - Question 4 - Rating Scale - Matrix 

Please indicate how often each of the following activities occurred throughout the 2009-10 school year. (Select one 
response per row). 


School procedures for SES sessions 

Always 

Often 

Sometimes 

Never 

Don't Know 

were organized in an efficient manner 
for SES-eligible students to attend 
their sessions as scheduled 

o 

o 

o 

o 

o 

I participated in the monitoring of 
2009-10 SES sessions. 

o 

o 

o 

o 

o 

SES sessions took place as scheduled. 

o 

o 

o 

o 

o 

SES sessions started on time. 

o 

o 

o 

o 

o 

SES sessions ended on time. 

o 

o 

o 

o 

o 


Page 1 - Question 5 - Choice - Multiple Answers (Bullets) 

Indicate any issues you had with the SES program implementation for SES students. (Select all that apply). 


□ None 

□ Staffing concerns 

□ Scheduling concerns 

□ Other, please specify 


Page 1 - Question 6 - Choice - Multiple Answers (Bullets) 

SES sessions took place at the following time(s). (Select all that apply). 


□ Before school 

□ After school 

□ On Saturdays 

□ During student's track-out 

Page 1 - Question 7 - Choice - One Answer (Bullets) 

What was the typical duration of an SES session at your school? 


O 1 hour 
O 1 .5 hours 
O 2 hours 

O Other, please specify 


Page 1 - Question 8 - Choice - One Answer (Bullets) 

With what frequency did SES sessions occur at your school? 


O Once a week 
O Twice a week 
O Other, please specify 
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Page 1 - Question 9 - Choice - Multiple Answers (Bullets) 

At what location were your school’s SES sessions heid? (Seiect all that apply). 


□ At the school site 

□ Away from the school site 

Page 1 - Question 10 - Open Ended - One Line 

What was the typical number of students in your SES students' group session. 


Page 1 - Question 1 1 - Open Ended - One Line 

How many students were in your SES program's largest group session? 


Page 1 - Question 12 - Open Ended - One Line 

How many students were in your SES program's smallest group session? 


Page 1 - Question 13 - Open Ended - Comments Box 

Please provide any additional comments you have regarding the 2009-10 SES Implementation. 


Page 1 - Question 14 - Choice - One Answer (Drop Down) 

At what school were you the site coordinator in 2009-1 0? 


O Brentwood Elementary 
O Fox Road Elementary 
O Poe Elementary 
O Wendell Elementary 
O York Elementary 


Page 1 - Heading 

This is the end of the survey. Please click the "Submit" button below in order to save your responses. You may review or 
change your responses prior to clicking "Submit." 


Thank You Page 

Thank you for your participation! 
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Appendix C 


2 ioom§rang 

2009-10 Supplemental Education Services (SES) Implementation 
Provider Survey 

Created: February 1 0 2011 , 1 0:1 8 AM 

Last Modified: February 21 201 1 , 9:01 PM 

Design Theme: Basic Blue 

Language: English 

Button Options: Labels 

Disable Browser “Back” Button: False 


2009-10 Supplemental Education Services (SES) lmplementationProvider&nbsp;Survey 



WAKE COUNTY 

ISITBUC SCHOOL SVCTEM 


Page 1 - Question 1 - Rating Scale - Matrix 


What percent of the time did each of the following occur when developing learning plans for SES student participants in 
2009-10? (Select one response per row). 


100% 

50% or more 

Fewer than 50% 

None 

Worked with WCPSS representatives 

O 

o 

o 

o 

Parents or guardians participated 

O 

o 

o 

o 

Used pre-assessment results 

O 

o 

o 

o 

Considered the student's individual 
needs 

Page 1 - Question 2 - Rating Scale - Matrix 

O 

o 

o 

o 

Please indicate how often each of the following activities occurred throughout the 2009-10 school year, 
response per row). 

(Select one 

SES pre- and post-assessments were 

Always 

Often 

Sometimes 

Never 

administered to students receiving 
services. 

o 

o 

o 

o 

Our SES sessions took place as 
scheduled. 

o 

o 

o 

o 

Our SES sessions started on time. 

o 

o 

o 

o 

Our SES sessions ended on time. 

We contacted parents of those students 
not attending SES sessions regularly. 

Page 1 - Question 3 - Choice - Multiple Answers (Bullets) 

o 

o 

o 

o 

o 

o 

o 

o 

With whom do you typically share each student's SES pre- 

and post-assessment results? (Select all that apply). 


□ Parents or guardians 

□ School site coordinator 

□ Teacher(s) 


□ School principal 

□ Other, please specify 
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Page 1 - Question 4 - Choice - Multiple Answers (Bullets) 

With whom do you typically share each student's SES progress reports? (Select all that apply). 


□ Parents or guardians 

□ School site coordinator 

□ Teacher(s) 

□ School principal 

□ Other, please specify 


Page 1 - Question 5 - Choice - Multiple Answers (Bullets) 

Who monitored your SES sessions? (Select all that apply). 


□ Provider administrator or designee 

□ School site coordinator 

□ WCPSS administrator 

□ NCDPI staff 

□ Other, please specify 


Page 1 - Question 6 - Yes or No 

Did you have any issues with staffing for the SES program implementation? 


O Yes 
O No 

O Please describe any issues. 


Page 1 - Question 7 - Yes or No 

Did you have any issues with scheduling for the SES program implementation? 


O Yes 
O No 

O Please describe any issues. 


Page 1 - Question 8 - Open Ended - Comments Box 

Please describe any other issues you had with the SES program implementation. 


Page 1 - Question 9 - Choice - Multiple Answers (Bullets) 

SES sessions took place at the following time(s). (Select all that apply). 

□ Before school 

□ After school 

□ On Saturdays 

□ During the SES student's track-out 
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Page 1 - Question 10 - Choice - One Answer (Bullets) 

What was the typical duration of an SES session? 

O 1 hour 
O 1.5 hours 
O 2 hours 

O Other, please specify 


Page 1 - Question 1 1 - Choice - One Answer (Bullets) 

With what frequency did SES sessions occur? 

O Once a week 
O Twice a week 
O Other, please specify 


Page 1 - Question 12 - Choice - Multiple Answers (Bullets) 

At what location were your school's SES sessions held? (Select all that apply). 


□ At the school site 

□ Away from the school site 

Page 1 - Question 13 - Open Ended - One Line 

What was the typical number of students in your SES students' group session? 


Page 1 - Question 14 - Open Ended - One Line 

How many students were in your largest group session? 


Page 1 - Question 15 - Open Ended - One Line 

How many students were in your smallest group session? 


Page 1 - Question 16 - Rating Scale - Matrix 

Please indicate your level of agreement with each of the following statements. (Select one response per row). 

Strongly Agree Agree Disagree Strongly Disagree 


School procedures for SES sessions 
were organized in a manner that 
enabled SES -eligible students to attend 
their sessions as scheduled. 

Attendance improved for those 
students not attending SES sessions 
regularly after we contacted their 
parents or guardians. 

Parents or guardians were given an 
opportunity to provide feedback 
regarding their satisfaction with their 
child's SES sessions. 


O 


O 


o 


o o 


o o 


o o 
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Page 1 - Question 17 - Yes or No 

Were there any participants who were not administered a post-assessment? 

O Yes 
O No 

O If "Yes," why? 


Page 1 - Question 18 - Yes or No 

Were any service plans changed during the implementation period? 

O Yes 
O No 

O If "Yes," why? 


Page 1 - Question 19 - Yes or No 

Were there any students who had attendance issues? 


O Yes 
O No 

O If "Yes," why? 


Page 1 - Question 20 - Open Ended - Comments Box 

Please provide any additional comments you have regarding the 2009-10 SES Implementation. 


Page 1 - Question 21 - Choice - One Answer (Drop Down) 

Please select the name of your organization which provided SES in 2009-10. 


O Academic Achievers/S&L Consultants 
O Academics Plus, Inc. 

O Measurement Incorporated (Accelerated Achievement) 

O Achieve Success Tutoring by University Instructors 
O AIM by Salient Learning 
O Allied Academics 
O ATS Project Success 
O It's Simply English 
O Learning & You 

O Master Mind Prep Learning Solutions, Inc. 

O Shaw University/Historically Minority Colleges & Universities Consortium of NC 
O Sterling Learning Center, Inc. 

O Sylvan Learning Center / Ace It 
O Village Learning Solutions 


Page 1 - Heading 

This is the end of the survey. Please click the "Submit" button below in order to save your responses. You may review or 
change your responses prior to clicking "Submit." 
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Appendix D 

SES Implementation Responsibilities 


Entity 

Category 

Details 

LEA 

A. Identify eligible students 

1 . All students receiving free or reduced priced lunches are eligible 

2. Students not receiving free or reduced priced lunches may receive SES, but funds spent on providing SES to 
non-low-income students toward meeting its 20 percent obligation may not be used. 


B. Notify parents about SES 
availability 

1 . Content of notice 

a) Explain how parents can obtain SES for their child 

b) Identify each approved SES provider within the LEA or in its general geographic location, including 
providers that are accessible through technology, such as distance learning 

c) Describe briefly the services, qualifications and evidence of effectiveness for each provider 

d) Indicate providers that are able to serve students with disabilities or LEP students 

e) Explain benefits of receiving SES 

f) Describe the procedures and timelines that parents must follow to select a provider to serve their child 

(1) where and when to return a completed application 

(2) when and how the LEA will notify parents about enrollment dates and start dates 

(3) whom to contact for more information 

(4) how priorities will be set in order to determine which eligible students receive services if insufficient 
funds to serve all eligible students 

2. Format of notice 

a) Easily understandable, in a uniform format, including alternate formats upon request, and to the extent 
practicable, in a language the parents can understand 

b) Clear and concise, and clearly distinguishable from other information on school improvement 

3. Help parents choose a provider, if requested 


C. Identify eligible students 
that will be receiving 
services 

1 . Apply fair and equitable procedures for serving students if not all students can be served 

a) Cut-off scores - lowest achievers in all grade levels 

b) Specific identified grade levels - lowest achievers 

c) Specific identified subject areas - lowest achievers 

2. Ensure that participating students with disabilities and LEP students receive appropriate services 

3. Ensure parents requesting service are aware of SES logistics regarding time period, session lengths, location, 
provider qualifications (subject areas, use of a variety of instructional methods, SWD and ESL accommodations, 
etc.) 

4. Prominently display on its Web site current information: 

a) number of eligible students 

b) number of participating students 

c) list of approved providers and the locations where services are provided 


D. Meet 20% funding 
obligation 

1 . SES provider programs 

2. Transportation 

3. Parent outreach and assistance 


Data Source: U.S. Department of Education (2009). No Child Left Behind: Public School Choice Non-Regulatory Guidance. Retrieved January 20, 2011, from 
http://www2.ed.gov/policy/elsec/guid/suppsvcsguid.doc. 
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Appendix D continued 
SES Implementation Responsibilities 


LEA 

(continue 

d) 

A. Enter into an agreement 
with a provider selected 
by parents of an eligible 
student 

1. Provisions regarding payments for services: 

a) frequency of payments 

b) may address missed sessions 

c) may be based on student attendance 

2. Provision prohibiting the provider from disclosing to the public the identity of any student eligible for or 
receiving SES without the written permission of the student’s parents 

3. Assurance that SES will be provided consistent with applicable health, safety, and civil rights laws 

4. Develop specific achievement goals for the student in consultation with the provider and the student’s parents 

5. Describe how student progress toward achievement goals will be measured and how the student’s parents and 
teachers will be regularly informed of that progress 

6. Create timetable for improving the student’s achievement 

7. Provisions for terminating a provider’s services: 

a) to specific students if the provider fails to meet those student’s achievement goals and timetables 

b) if the provider violates its agreement regarding 

(1) invoicing of payments for services 

(2) maintaining student privacy 

(3) adherence to applicable health safety, and civil rights laws 

(4) meeting other terms such as background checks of LEA contractors 

(5) providing student progress reports regularly 


B. Ensure that parents are 
notified by the provider 
if their child is not 
attending regularly. 


C. Obtain parent feedback 
on satisfaction with SES 
service. 


Provider 

A. Attend to logistics of 
agreement with LEA 

1. Provide SES to eligible students receiving services until the end of the school year based on available funds and 
service intensity 

2. Agree with school and parents on a schedule for informing parents and the student’s teacher(s) about the 
student’ s progress 

3. Provide SES consistent with applicable health, safety, and civil rights laws 

4. Provide SES that are secular, neutral, and non-ideological 

5. Conduct scheduled sessions 

6. Start and end SES sessions on time 

7. Maintain appropriate group sizes for instruction 


Data Source: U.S. Department of Education (2009). No Child Left Behind: Public School Choice Non-Regulatory Guidance. Retrieved January 20, 2011, from 
http://www2.ed.gov/policy/elsec/guid/suppsvcsguid.doc. 
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Appendix D continued 
SES Implementation Responsibilities 


Provider 

(continue 

d) 

A. Provide instruction based 
on agreement with LEA 

1 . Administer pre/post assessments 

2. Work with LEA and parents in development of learning plan 

a) Consistent with the instructional program of the LEA 

b) Consistent with SWD lEP of participating students 

c) Consistent with Section 504 needs of participating students 

d) Consistent with LEP needs of participating students 

e) Aligned with State academic content and student academic achievement standards 

3. Adhere to learning plan 

4. Enable the student to attain his or her specific achievement goals 

a) Provide high quality, research-based instruction (e.g., differentiation) that is focused, intensive, and targeted 
to student needs and designed to increase student academic achievement 

b) Provide constant and systematic feedback to students on what they are learning 

5. Measure the student’s progress according to agreed-upon schedule 

6. Regularly inform the student’s parents and teachers of student progress 

7. Inform parents if child is not attending sessions regularly 

8. Measure whether participating students and their parents are satisfied with the instructional program 

Parent 

B. Select a provider from 
the State-approved list 
that is in or near the 
LEA. 


C. With the LEA and the 
selected provider, must 
develop and identify 
specific academic 
achievement goals for 
the student, measures of 
student progress, and a 
timetable for improving 
achievement. 


D. Ensure that their child 
attends the SES sessions 
in which he or she is 
enrolled. 


E. Other desired 
responsibilities 

1 . Support provider and LEA efforts to work with their child toward attaining achievement goals 

2. Encourage their child to succeed 

3. Provide feedback on satisfaction with SES services 


Data Source: U.S. Department of Education (2009). No Child Left Behind: Public School Choice Non-Regulatory Guidance. Retrieved January 20, 2011, from 
http://www2.ed.gov/policy/elsec/guid/suppsvcsguid.doc. 
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Appendix E 

Title I School Improvement Provisions under No Child Left Behind Legislation 
Implementation Timeline -Supplemental Educational Services Round 1 


Date 

What 

Explanation 

Notes 

September 2009 

Communication 
with Principals 

Plan for provider fair 

Identify provider fair dates 

September 18, 2009 

Provider list 
updated 

Final list of service providers posted on the 
DPI honnepage 

Local superintendents. Title 1 directors and others review 
provider list. Superintendents and Title 1 directors notified of the 
posting 

September 21, 2009 

Provider Meeting 

Discuss parameters and arrangements for 
services and provider fair 

Tights and loose of SES 

September 24, 2009 

Principal Meeting 

Discuss parameters and arrangements for 
services and provider fair 

Tights and loose of SES 

September 30, 2009 

Notification to 
parents 

SES letter sent 

Letter will state: 

• Potential for schools offering SES 

• Students may be eligible 

• Parents' responsibility in process 

October 5-16, 2009 

Parent/ Provider Fair 

Each school will hold provider fair for parents 

Parents select provider for services 

October 26-30, 2009 

Complete contracts 

WCPSS will enter into contract agreements 
with providers 

Each student contract will 

• be based on the identified needs 

• Outline sessions 

• Provide for assessment information and progress 
reports 

October 30, 2009 

Deadline for SES 
requests 

SES requests must be received 


November 2009 

Notify parents of 
assigned provider 

Send letters to parents with contact info of 
provider 


Meeting with 
providers and 
parents 

Providers and parents discuss educational 
plan 

Provider must develop educational plan with parent participation 

Update provider 
rosters 

Send rosters to site coordinators and 
providers 


November 16, 2009 

Start services 

Provider will begin services 

Provider will send monthly reports to site SES Coordinator, the 
Title 1 director and parent 

November 2009 

Calendars and 
Schedules 

Providers will receive detailed calendar for 
each site 

Calendars will include projected end dates 


Data Source: WCPSS Title I Department 
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Appendix E continued 

Title I School Improvement Provisions under No Child Left Behind Legislation 
Implementation Timeline -Supplemental Educational Services Round 2 


Date 

What 

Explanation 

Notes 

December 2010 

Round 2 
Notifications 

Send letters about round 2 enrollment period 
to parents and display at schools 

Timeline for requests 

January 4-15, 2010 

Parent requests 

Parents select providers to start round 2 of 
tutoring 


Notify parents of 
assigned provider 

Send letters to parents with contact info of 
providers 


Week of J anuary 18, 
2010 

Meeting with 
providers and 
parents 

Providers and parents discuss educational 
plan 

Provider must develop educational plan with parent participation 

J anuary 2010 

Update provider 
rosters 

Send rosters to site coordinators and 
providers 


Week of February 1, 
2010 

Start services 

Provider will begin services 

Provider will send monthly reports to site SES Coordinator, the 
Title 1 director and parent 

February 2010 

Calendars and 
Schedules 

Providers will receive detailed calendar for 
each site 

Calendars will include projected end dates 


Data Source: WCPSS Title I Department 
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Parti: 


WAKE COUNTY 

PUBLIC SCHOOL SYSTEM 


Supplemental Educational Services 
Personalized Education/ Focused Learning Plan 
Descriptive I nformation 


SCHOOL COMPLETES 


Student Name 

Student Birth Date 
M/D/Y 

Teacher Name 

Reading 

Math 

School 

Elementary 

Student 1 D # 

Parent / Guardian Name 

Home Telephone 

Work Telephone 

Student Race (Circle) 

Student Gender 

□ Female 

□ Male 

Student Language 

□English Speaking 
□Non English Speaking 

List language, if 
other than English 

Asian 

American Indian 

Black 

Hispanic 

Multi-racial 

White 

Other (list) 

Student Address 


SCHOOL COMPLETES 


□ Exceptional Child 

□ AIG 

□ 504 

□ Prior Retainee and 
Grade Retained 

n other: 

n LEP 

□ Regular Education 



Please list I EP goals if applicable 


Reading lEP Goals 


Math lEP Goals 



Part 1 1 : Modification! s) for I nstruction: ( Select all appropriate) SCHOOL COMPLETES 



Modified 1 nstruction 

Comments 

□ 

visual examples 


□ 

modify length of assignment 


□ 

divide task into parts 


□ 

concrete instruction (hands on) 


□ 

mark in book 


□ 

extended time 


□ 

oral tests 


□ 

use of a dual language dictionary 


□ 

ESL class 


□ 

EC services 


□ 

Other 



Learning Plan, Updated August 09 
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Appendix F continued 
WCPSS Learning Plan Template 2009-10 

Part III: Diagnostic I nformation SCHOOL COMPLETES 


Area 

Comments 

Performance Level 

Reading 



Math 




Part IV: Results of Provider Diagnostic I nformation PROVIDER COMPLETES 


Assessments 

Pre Test Score/ Date 

Post Test Score/ Date 

Comments 


















Part V: Provider I nstructional Materials PROVIDER COMPLETES 



Provider 1 nstructional 
Materials/ Programs 

1 nterventions that will be used 

□ 



□ 



□ 



□ 



□ 




Part VIII: Timeline PROVIDER, PARENT, PRINCIPAL, TEACHER COMPLETES 


Length/ number of sessions to be provided 

Session length in minutes: 

Number of weekly sessions: 
Total number of sessions: 

Dates of sessions 

Beginning date: 

Ending date: 

Days of the week tutoring will occur: 

Location of services 

Services will be provided in the following location: 


I have reviewed the Learning Plan Agreement. I agree to the statement of goals and timeline stated in this 
agreement. I have been given the opportunity to participate in the development of this plan. 

Signatures indicate agreement with the Learning Plan Agreement 


X 

Provider Signature and Company Name Date 

X 

Parent Signature Date 

X 

Teacher Signature Date 

X 

Principal Signature Date 


Learning Plan, Updated August 09 
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Appendix G 

SUPPLEMENTAL EDUCATI ONAL SERVI CES MONITORI NG TOOL 


Name of Observer Date 

Activity: Announced/ Unannounced Visit Begin Time End Time. 


Name o f SES Provider 

Name of SES Schools 




MANAGEMENT 

YES 

NO 

NOT OBSERVED 

1. Provider's site coordinator or designee is present before the delivery of 
services and assumes responsibility for students until tutoring begins. 




2. Provider's site coordinator or designee is available to monitor program 
implementation and has no instructional responsibilities. 




3. Provider's site coordinator or designee is present during the delivery of 
services. 




4. Provider's site coordinator or designee assumes responsibility for students 
after the delivery of services and remains until all participants are picked up 
by an authorized person 




5. Time for snacks, breaks, or incentives is provided outside of the contracted 
time for the tutoring session. 




6. The number of students in the group meets contractual specifications. 




7. All assigned tutors are present. 




NOTES/OBSERVATIONS 


Number of Students Observed: 
Name of Tutor Observed: 

Reading 

Math 

Location 

1 nstructional Methods 

YES 

NO 

NOT OBSERVED 

8. The tutor is prepared and begins instruction on time. 




9. The tutor continues to provide instruction throughout the contracted 
time for the tutoring session. 




10. The tutoring session is focused on instruction. 




11. Instructional materials are available for all students. 




12. The instructional materials are matched to the needs/level of the 
students as determined by the pre-assessment. 




13. Tutors give consistent verbal and/or written feedback to students 
on what they are learning. 




14. Instruction is aligned with the NCSCOS. 




15. The tutor uses appropriate high guality instructional strategies 
which are matched to the needs of the student. 




16. The instructor provides services that are secular, neutral, and non- 
ideological in content. 




NOTES/OBSERVATIONS 



October 2009 


Wake County Public Schools 
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Appendix H 

Results of NC Supplemental Educational Services (SES) 

Parent Satisfaction Survey - Questions 15-21 -Wake County Public School System 

Questions 15-21 pertain to parent satisfaction with the school district in terms of connnunication and the provider selection process. Means and standard deviations are 
calculated based on a 4-point Likert-type scale with the following values: l=Strongly Disagree, 2=Disagree, 3= Agree, 4=Strongly Agree. Calculations are made separately 
for each session and a combined score is also provided. Cells indicate frequency of response, followed by percentage of response in parentheses. 


Total A=1 71 Results for a particular question may not always sum to 100% due to non-respondents. 



Questions 

Indicate how much you agree or disagree with the 
following items about the school district: 

Mean 

Stand 

Dev 

Strongly 
Disagree (1) 

Disagree (2) 

Agree (3) 

Strongly 
Agree (4) 

Don't Know 

N/A 

n 

% 

n 

% 

n 

% 

n 

% 

n 

% 

n 

% 

15 

1 was notified by my child's school about free 
tutoring... 

3.56 

0.66 

5 

2.9% 

_ 

_ 

58 

33.9% 

102 

59.6% 

2 

1.20% 

2 

1.2% 

16 

The district provided the necessary information to 
help me select a tutoring provider... 

3.48 

0.69 

4 

2.3% 

6 

3.50% 

62 

36.3% 

93 

54.4% 

4 

2.30% 

_ 

_ 

17 

1 was given enough time to decide which tutoring 
provider 1 wanted for my child... 

3.39 

0.76 

6 

3.5% 

9 

5.30% 

61 

35.7% 

83 

48.5% 

8 

4.70% 

3 

1.8% 

18 

The district returned phone calls, addressed 
questions pertaining to free tutoring in a timely 
manner... 

3.25 

0.78 

5 

2.9% 

14 

8.20% 

63 

36.8% 

60 

35.1% 

9 

5.30% 

16 

9.4% 

19 

1 am happy with the way my school district helped 
to get free tutoring for my child... 

3.53 

0.64 

3 

1.8% 

4 

2.30% 

59 

34.5% 

95 

55.6% 

5 

2.90% 

2 

1.2% 

20 

1 feel free tutoring improved my child's academic 
skills this year... 

3.42 

0.67 

3 

1.8% 

7 

4.10% 

69 

40.4% 

79 

46.2% 

8 

4.70% 

2 

1.2% 

21 

1 would have my child participate again in free 
tutoring next year... 

3.69 

0.62 

4 

2.3% 

1 

0.60% 

37 

21.6% 

120 

70.2% 

6 

3.50% 

2 

1.2% 


Williams, M., Meli, J., Poole, A., & Amwake, L. (2010). NCDPI supplemental educational services (SES) 2009-2010 evaluation report. Raleigh, NC: 
North Carolina Department of Public Instruction. Retrieved January 7, 2011, from http://www.ncpublicschools.org/docs/program- 
monitoring/titleIA/ses/evalreports/ses09- 1 0.pd 
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